Cross-Domain Multitask Model for Head Detection and Facial Attribute Estimation

Extracting specific attributes of a face within an image, such as emotion, age, or head pose has numerous applications. As one of the most widely used vision-based attribute extraction models, HPE (Head Pose Estimation) models have been extensively explored. In spite of the success of these models, the pre-processing step of cropping the region of interest from the image, before it is fed into the network, is still a challenge. Moreover, a significant portion of the existing models are problem-specific models developed specifically for HPE. In response to the wide application of HPE models and the limitations of existing techniques, we developed a multi-purpose, multi-task model to parallelize face detection and pose estimation (i.e., along both axes of yaw and pitch). This model is based on the Mask-RCNN object detection model, which computes a collection of mid-level shared features in conjunction with some independent neural networks, for the detection of faces and the estimation of poses. We evaluated the proposed model using two publicly available datasets, Prima and BIWI, and obtained MAEs (Mean Absolute Errors) of 8.0 ± 8.6, and 8.2 ± 8.1 for yaw and pitch detection on Prima, and 6.2 ± 4.7, and 6.6 ± 4.9 on BIWI dataset. The generalization capability of the model and its cross-domain effectiveness was assessed on the publicly available dataset of UTKFace for face detection and age estimation, resulting a MAE of 5.3 ± 3.2. A comparison of the proposed model’s performance on the domains it was tested on reveals that it compares favorably with the state-of-the-art models, as demonstrated by their published results. We provide the source code of our model for public use at: https://github.com/kahroba2000/MTL_MRCNN

Evaluation of 3D obstacle avoidance algorithm for smart powered wheelchairs

This research investigates the feasibility for the development of a novel 3D collision avoidance system for smart powered wheelchairs operating in a cluttered setting by using a scenario generated in a simulated environment using the Robot Operating System development framework. We constructed an innovative interface with a commercially available powered wheelchair system in order to extract joystick data to provide the input for interacting with the simulation. By integrating with a standard PWC control system the user can operate the PWC joystick with the model responding in real-time. The wheelchair model was equipped with a Kinect depth sensor segmented into three layers, two representing the upper body and torso, and a third layer fused with a LIDAR for the leg section. When using the assisted driving algorithm there was a 91.7% reduction in collisions and the course completion rate was 100% compared to 87.5% when not using the algorithm

Artificial intelligence for safe assisted driving based on user head movements in robotic wheelchairs

Wheelchairs users don’t always have the ability to control a powered wheelchair using a normal joystick due to factors that restrict the use of their arms and hands. For a certain number of these individuals, which still retain mobility of their head, alternative methods have been devised, such as chin-joysticks, head switches or sip-and-puff control. Such solutions can be bulky, cumbersome, unintuitive or simply uncomfortable and taxing for the user. This work presents an alternative head-based drive-control system for wheelchair users

Reinforcement Learning for Shared Autonomy in Powered Wheelchair Navigation

Assistive robotics is witnessing a surge in research focusing on designing algorithms and frameworks that offer personalized support to users, considering their intentions and adapting system responses accordingly. This thesis delves into the integration of artificial intelligence in assistive technologies for powered wheelchairs, with a primary emphasis on the challenging problem of shared control through reinforcement learning. Shared control, also known as shared autonomy, has been extensively studied, especially in the context of powered wheelchairs. Many wheelchair users rely on aid to enhance their everyday autonomy, particularly those who cannot use conventional joystick control interfaces, often encountering frustrations, fatigue, and compromised safety. Existing shared control methods typically involve blending human and autonomous controller decisions or predicting user goals to act autonomously. Unfortunately, such approaches often rely on assumptions like known goal sets, world dynamics models, and user behavior models, which limit adaptability. Motivated by the shortcomings of prior approaches and inspired by recent machine learning advances, this thesis introduces a novel shared control method using deep reinforcement learning within a continuous action space, which lifts the reliance on the aforementioned assumptions. Initially, a reinforcement learning agent is developed to autonomously navigate complex indoor environments without the need for a map. The agent is trained using a virtual robotic wheelchair and rigorously validated against popular path planning methods. Subsequently, artificial noise is injected into the learned model to simulate disabled user input, enabling the training of an end-to-end shared control system. A modification in the typical reinforcement learning objective ensures compliance with user intentions while simultaneously maximizing future rewards associated with the assistive nature of the system. The shared control system receives noisy user commands and sensor data to generate corrective control commands for the wheelchair. Rigorous simulations and real-world trials with human users demonstrate significant reductions in collisions and increased obstacle clearance, albeit with a trade-off in user satisfaction. Additionally, this thesis presents a non-intrusive, vision-based head-control interface for powered wheelchairs, employing face detection and head pose estimation. Through human user trials, the effectiveness and performance of this interface are benchmarked, confirming its viability as an alternative to the standard joystick interface. Notably, when combined with the shared control system in further real-world trials, the proposed assistive system proves adept at compensating for the less accurate input of this more challenging interface, resulting in a remarkable 92\% reduction in collisions and improved overall adequacy. In summary, this thesis introduces a mapless autonomous navigation method for powered wheelchairs, a novel shared control framework employing deep reinforcement learning, and a non-intrusive vision-based head-control interface. The proposed assistive system is empirically validated, showcasing its substantial impact on enhancing user autonomy and safety in powered wheelchairs